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Abstract. /n the PISA scientific literacy 
monitoring carried out in 2006 and 2009 
Slovak students achieved results below the 
OECD average scores while Czech students 
achieved significantly better results. Fol- 
lowing the PISA results there was carried 
out a research aimed at evaluation of the 
current state of computer science educa- 
tion/informatics teaching at an upper 
secondary school level in both countries 
and moreover also in Belgium. In all three 
concerned countries a lack of textbooks of 
an appropriate quality was identified as a 
main weakness. Further identified weak- 
nesses were: in Slovakia a problem of a 
lack of tasks attractive for students, in the 
Czech Republic a weak attractiveness of 
the teacher's presentation of new material 
for students and identically to Belgium a 
lack of engagement of students in tasks 
solved in class. Positively evaluated was: in 
Slovakia attractiveness of the curriculum 
and methods of new material presentation 
by teachers, and in the Czech Republic and 
Belgium clarity of the teacher's presenta- 
tion of new curriculum. 
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RESULTS OF PISA AND 
EVALUATION OF COMPUTER 
SCIENCE EDUCATION 


Jan Zahorec, 
Alena Haskova, 
Michal Munk, 
Martin Bilek 


Introduction 


Since 2006 the scientific literacy has been a part of the PISA 
international monitoring of school achievements (Programme for 
International Student Assessment). In three year cycles the OECD 
study PISA tries to bring new information on the level of scientific 
literacy in the OECD member countries and other participating 
countries, based on testing the skills and knowledge of 15-year- 
old students. Within this monitoring the scientific literacy is 
defined as the ability to use scientific knowledge and processes 
not only to understand the natural world but also to participate 
in decisions that affect it. 

In both carried out cycles of the PISA scientific literacy 
monitoring (2006, 2009) Slovak students achieved results, which 
were below the OECD average scores (OECD 2007a, 2007b; OECD 
2010). To compare with the year 2006, in 2009 the average score 
of the Slovak students rose insignificantly up to 2 points and 
reached 490 value at the average OECD score 501. In the rank- 
ing of the participating OECD countries this meant 23" - 29" 
place and among the all 65 PISA participating countries it was 
29" — 37" place. On the one hand this evoked a great discussion 
on dissatisfactory with the PISA results and the quality of natural 
science teaching at both primary and secondary schools. But on 
the other hand in practice the position of natural science teach- 
ing became worse than it had been: a school reform decreased 
the number of compulsory natural science lessons per week in 
newly created State educational programme and some further 
cuts of the relevant subjects have been done at schools through 
their school educational programmes. 

Although the scientific literacy cannot be identified with 
the computer or digital literacy, nowadays one has to be com- 
puter/digital literate to develop his/her scientific literacy (Luu 
& Freeman, 2011). Computer literacy can be understood as a 
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prerequisite or a component part of the scientific literacy. The question of the relation of scientific 
and computer literacy can be perceived also as a consequence of the issue of computer science in- 
corporation among natural or technical sciences (Alexandrov, Alexandrov, & Ramirez, 2012). Usually 
computer science is not perceived as a part of natural sciences (in this sense as not equal to biology, 
chemistry, physics neither mathematics) but in its nature it is a subject of technical and mathematical 
nature. Moreover, as it has been already mentioned, it creates a platform and at the same time it is 
an integral part of the technical education (De Jong, 2007; Hus & Abersek, 2007; Vishkurti, Spahiu, & 
Paci, 2011). This fact is reflected also in the four areas on which the scientific literacy (or specifically 
scientific knowledge) PISA monitoring was aimed at: the role of science and technology, relation be- 
tween science and technologies, basic notions and important principles (OECD 2010, Olsen, Prenzel, 
& Martin, 2011). 


Research Problem 


The unsatisfactory results of Slovak students in the two cycles of PISA monitoring (2006, 2009) 
and the issue of computer education as an integral part of technical education or as a platform for 
one’s scientific literacy development evoked an intention to carry out a research aimed at evaluation 
of quality of informatics teaching in the Slovak Republic and its comparison with the situation in 
other countries. As PISA assesses not only how far students near the end of compulsory education 
have acquired the relevant knowledge and skills (in terms of mastery of the school curriculum) but 
also how able they have been to use them in practice and for full participation in society (in terms of 
important knowledge and skills needed in adult life), it was decided to set the intended assessment 
on the students’ point of view of the related issues. 


Methodology of Research 
General Background of Research 


The main goal of the research was to evaluate the current state of teaching informatics at the 
upper secondary school level of the system of education in the Slovak Republic (SK) (ISCED 3A; 16 - 
19 year age students). Additionally to this it was decided to evaluate, in the same way and based on 
the same criteria, also the situation in the Czech Republic (CZ) and Belgium (BE). 

The Czech Republic was chosen because of two reasons. One reason was the common roots 
and development of the both national systems of education in frame of previous Czechoslovakia 
(1914 —- 1992). The second reason was the results of PISA monitoring which showed that the Czech 
Republic had significantly better results than the Slovak Republic. Based on the results of the PISA 
monitoring (Paleckova et al., 2007), Czech students belong to ones with above-average results in the 
science test (mean OECD - 498 points, Czech students — 513 points, Slovak students — 488 points) as 
well as in the mathematical part of the test (mean OECD — 500 points, Czech students — 510 points, 
Slovak students — 492 points) (Paleckova &Tomasek, 2005; Fryzkova & Paleckova, 2007; Paleckova, 
Tomasek, & Basl, 2010). 

Belgium, as another comparative country with which the situation in Slovakia was compared, 
was chosen in regard to accessibility. 

Due to the diversity of secondary schools and with this diversity connected wide range of 
specific implementation of computer science education, we focused our attention on a single type 
of school in this category: in Slovakia and the Czech Republic on grammar schools, and in Belgium 
on general secondary schools (ASO) representing an equivalent to them. In these schools computer 
science education is based mainly on the school subject informatics taught as a compulsory subject 
aimed at acquisition of a “general informatics education’, i.e. at acquisition of basic knowledge and 
skills to work with digital technologies to search, process and present information which is necessary 
to solve different tasks. 
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Instrument and Procedures 


The quality of teaching a school subject is usually assessed on the basis of its curriculum con- 
tent analysis and students learning outcomes. We decided to set the evaluation of current state of 
informatics teaching on students opinions and their evaluations of this state. The evaluation was 
done based on following 14 selected specific factors gauging the students’ point of view: 

P1 — the popularity of the subject, 

P2 — the applicability of the gained knowledge in the students’ future, 

P3 - attractiveness of the curriculum content, 

P4 — the demands of the curriculum, 

P5 — the clarity of presentation of new material, 

P6 - the attractiveness of curriculum presentation by teachers, 

P7 — the suitability of particular methods for curriculum presentation, 

P8 — the engagement level of tasks to be solved, 

P9 — the clarity of textbooks used, 

P10 — the usability of knowledge for solving practical problems, 

P11 — the attractiveness of teaching aids used, 

P12 - the way in which students make written notes of the presented subject matter, 

P13 - the appropriateness of specific methods in written notes preparation, 

P14 — sources of concern related to the subject. 

Specification of the factors resulted from our previous experiences and research results (Haskova, 
2003; Zahorec, HaSkova, & Munk, 2010), professional literature in this area (Vari, 1997; Hrabal, 1989; 
Seben & Jakubov, 1997; Balint & Nogova, 2003; Ross & Genevois, 2006; Ribeiro & de Gusmao, 2010) 
and consultations with other experts (both research workers and computer science teachers from 
various types of schools and various practice duration). The data collection was carried out using 
a 14-item questionnaire, in which its particular items corresponded with the above-mentioned 
investigated factors P1 — P14 corresponding with the quality of computer science education/infor- 
matics teaching. 

In the used questionnaire the respondents expressed their answers (evaluations of the relevant 
assessed factors) according their opinions to the ordinary items P1, P2, P3, P4, P5, P6, P8, P9 and P10 
using a 7-point scale (1 —- most negative assessment/attitude; 4 — neutral, emotionally indifferent 
assessment/attitude; 7 — most positive assessment/attitude). 

In the nominal questionnaire items P7 and P11 — P14 the respondents were asked to choose 
among several given alternatives the most suitable answer for them. The offered alternatives to the 
particular items were these following: 

P7 — the kind of explanation preferred by the student: a — teacher explains the subject matter 
without using visual teaching aids; b - teacher explains the subject matter using various 
teaching aids; c —- teacher involves also students in the explanation of the new subject matter; 
d — teacher gives individual tasks to students and supervises their progress; e — if other, state 
what you like; 

P11 - positive or negative answer to the question whether there are used also other teaching aids 
during the lessons of informatics in addition to computers (P11a) and in case of a positive 
answer evaluations how interesting the used teaching aids are for the students (P11b): 
a -very uninteresting; b —- uninteresting; c — rather uninteresting; d — neither uninteresting nor 
interesting; e - rather interesting; f — interesting; g — very interesting; 

P12 - the way in which students are asked to make their written notes: a — the teacher dictates 
us the notes; b - we do our written notes according the teacher who writes the notes on the 
blackboard or screens the notes in an electronic way using a data projector; c — a part of the 
written notes we do according the notes made by the teacher and a part of the notes we do 
from the textbooks ourselves; d- we make our written notes completely ourselves on the basis 
of the teachers’ explanation; e — we make all our written notes from the textbook ourselves at 
school; f - we make all our written notes from the textbook ourselves at home; g —- we do not 
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make written notes of the presented subject matter at all; 

P13- positive or negative answer to the question whether the used way of taking the notes is 
convenient for the student (P13a) and in case of a negative answer indicating the one, from 
the same alternatives as stated in P12, which would the student prefer more (P13b); 

P14- possible reasons to feel scared or nervous before the informatics lessons: ag—/am not used 
to be afraid of anything; b — unpreparedness/I am not prepared properly; c — oral examina- 
tion; d - practical tests; e —- getting a bad mark; f - fear of repeated lack of understanding the 
presented subject matter; g - other, state what. 

The questionnaire reliability was proved through identification of its suspicious items by the 
means of the analysis of the questionnaire/item analysis. The final total questionnaire reliability 
a = 0.84, calculated using the Cronbach alpha, indicates a high inner consistence of this measuring 
tool as a tool for relevant reliable research data obtaining. 

Data obtained for each questionnaire item were processed in a dependence on the factor 
COUNTRY, GENDER and combination of the factors COUNTRY*GENDER. Statistical processing of the 
obtained data was based on the use of chi-square test, contingency coefficient, analysis of variance 
for repeated measurements ANOVA, Greenhous-Geisser and Huynh-Feldt corrections for repeated 
measures in the analysis of variance and graphical visualization. 


Sample of Research 


The research sample of the upper secondary school level (ISCED 3A; 16 — 19 year age stu- 
dents) was limited to 4-year and 8-year grammar schools (SK and CZ) and to them relevant general 
secondary ASO schools (BE — Algemeen Secundair Onderwijs). Research samples in the observed 
countries were, due to our limited opportunities, based on the availability of the schools. Even though, 
the schools from the Slovak and Czech Republic involved in the research (SK - 8 different grammar 
schools, CZ — 6 different grammar schools) represented various regions as well as different sizes of 
towns. Only the Belgium research sample cannot be taken as a representative one, as the respon- 
dents involved in research have represented only one school. The total number of the respondents 
was 368 (SK — 246, CZ — 70, BE - 52) from which 222 were boys (SK — 154, CZ — 39, BE - 29) and 146 
girls (SK - 92, CZ - 31, BE - 23). 


Ordinary/Scale Items Analysis 


Analysis of variance for repeated measures (ANOVA repeated measures) was used in tests of 
differences between two or more dependent samples — repeated measures. Analysis of variance 
repeated measures was used when, apart from the factor which determines the repeated measure 
at particular objects - dependent samples (case of the items P1 — P6, P8 — P10), there are also fac- 
tors (COUNTRY, GENDER) which divide the objects into several groups — independent samples. If the 
independent samples are compared, this factor is called between-groups factor (COUNTRY, GENDER). 
If the dependent samples are compared — repeated measures of the same objects, the factor is called 
within-group factor (ITEM). Calculations of the tests of significance are different for these different 
types of factors but the interpretation of their results is the same (Munk, 2011). 

The model of the analysis of variance with repeated measure without any group distribution 
is described by the formula: 

VY =P, +), tA, Fe, 


where Z/ is a mean of the dependent variable, a, is the effect of the j-th repeated measure, 7 is 
the effect of the i-th object, Ai, is an interaction between the repeated measure j and the object i. 

We are interested in the null hypothesis, which is: 

HO: ws =...= 4, (the means of the repeated measures are equal). 

For the use of the tests following preconditions have to be fulfilled: 

normal distribution of the dependent variable in the groups according the factor levels, 
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precondition of the covariant matrix assumption of sphericity — equation of the variances and 
covariances in the covariant matrix for the repeated measures. 


Nominal Items Analysis 


To find out relations between two nominal variables, the contingence analysis was used, i.e. the 
contingence analysis was used to analyse nominal items dependences (P7, P11a, P12, P13a, P14 x 
COUNTRY, GENDER). The group of the distribution-free tests belonging to the contingence analysis is 
based on the contingence table. These tests verify the null hypothesis, which states that the variables 
are independent: 

HO: P(X a Y) = P(X )P(Y opposite to the alternative X, Y are dependent. 

For the purpose of the more-field table analysis the chi-square test of independence was used. The 
chi-square test of independence represents enhancement of the chi-square test of goodness of fit and 
results from the contingence table of the observed frequencies, where the observed frequency dj is 
the frequency of the combination x; A J ,. 


V.S . 
-—/ are those which fulfil the null hypothesis on the two variables 





The expected frequencies e,= 


n 
independence. The expected frequency of the relevant field is equal to the quotient of the conjunction 
of the relevant observed frequency of the line and column and the total number of the observations. 


The chi-square test = Sy lana 


i=l j=l ei 
expected frequencies are stochastic (the variables are independent) or statistically significant (the 
variables are dependent). 

The chi-square test can be used only in the case when the expected frequencies are big enough. 
In a case when the expected frequencies are not big enough, the Fisher test can be used, but this 
is applicable only for the four-field tables. 

To assess the relation level between two nominal variables the contingence coefficients (Pear- 
son, Cramer V) were used. These are values from the interval <0, 1>, where 0 means no relation 
and 1 means an ideal relation. Their significance was tested by the means of the chi-square test 
of independence. (For more detailed information on the conceptual and methodological basis of 
the research and the pilot verification of the administered questionnaire reliability see Zahorec & 
Haskova, 2011.) 


verifies whether the differences between the actual and 


Results of Research 
Ordinary/Scale Item Research Results 


Table 1 shows summarized results of the analysis of repeated measures calculated for the 
ordinary items of the administrated questionnaire (P1 — P6 and P8 — P10). The analysis was used to 
test the effect of COUNTRY and GENDER, both as independent factors, as well as their interaction 
on the aggregate questionnaire score (score of the relevant items). The factors of COUNTRY reached 
a p-value of less than 0.01, which means that responses to the items are significantly affected by 
the factor COUNTRY and they are not affected by the factor GENDER nor by the interaction of 
COUNTRY*GENDER (p > 0.05). Despite the fact that differences in the responses of boys and girls 
are not statistically significant, figure 3 shows a tendency of boys to evaluate individual factors 
higher than girls. 
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Table 1. Repeated measures analysis of variance. 


Ss df MS F p 
Intercept 48096.23 1 48096.23 8235.616 0.0000 
Country 86.47 2 43.24 7.403 0.0007 
Gender 7.08 1 7.08 1.213 0.2714 
Country*Gender 0.03 2 0.01 0.002 0.9977 
Error 2114.09 362 5.84 


As there was proved a significant difference among the respondents’ answers in dependence to 
the factor COUNTRY, the question was between which groups there is the significant difference in the 
assessment of the particular items. The homogeneous groups according the factor COUNTRY were 
found out by the means of the multiple comparisons of the relevant couples. The results of this multiple 
comparisons are summarised in table 2. The results show that the first homogeneous group consists 
of the Czech and Belgian students and the second one of the Slovak and Belgian students. Within 
these groups the respondents’ responses to the particular questionnaire items did not differ, i.e. within 
these groups the answers were almost the same irrespective of the respondents’ GENDER (within the 
subgroups created according the COUNTRY factor). The answers to the ordinary items as a whole were 
rather positive. The highest final overall score (average score for all monitored items) was achieved in 
Slovakia (4.95) while the Czech Republic recorded the lowest score (4.51). Belgium shows final average 
results that are essentially similar to Slovakia’s (a score of 4.87). Statistically, a significant difference in 
the responses to the items P1 — P6 and P8 —- P10 as a whole was reached between the groups of Slovak 
and Czech respondents. The statistical significance of the remaining differences was not proved. 


Table 2. Homogeneous groups identification. 


Factor COUNTRY Average score 1 2 
Czech Republic 4.514 re 
Belgium 4.874 ek wee 
Slovak Republic 4.948 tee 


After investigation of the ordinary items total score, we tested which items have significant differ- 
ences in the assessment without differentiating the respondents into groups and in dependence on 
the followed factors (figure 1). The findings were verified by the means of the Greenhouse-Geisser and 
Huynh-Feldt corrections for repeated measures of the analysis of variance to eliminate the problems 
with the covariance matrix assumption of sphericity (table 3). The results of the testing of differences 
in respondents’ answers to individual items, based on the Greenhouse-Geisser and Huynh-Feldt cor- 
rections (Lower Bound) for repeated measures ANOVA, confirmed the statistical significance (p < 0.05) 
in relation to the factor of COUNTRY. 
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Figure 1; Average score and confidence interval of the ordinary items according to a combination 
of the COUNTRY and GENDER factors. 


Table 3. Greenhouse-Geisser and Huynh-Feldt corrections (Lower Bound) for repeated measures 


ANOVA. 
Lower Bound Lower Bound Lower Bound Lower Bound 
Epsilon Adjusted df1 Adjusted sv2 Adjusted p 
ITEM 0.1250 1.0000 362.0000 0.0000 
ITEM*COUNTRY 0.1250 2.0000 362.0000 0.0000 
ITEM*GENDER 0.1250 1.0000 362.0000 0.1194 
ITEM*GENDER*COUNTRY 0.1250 2.0000 362.0000 0.2630 


The graph in figure 2 shows results obtained for each ordinary item (the average score and inter- 
val score of the average assessment of the items) separately for groups of Slovak, Czech and Belgian 
respondents. Figure 3 shows results obtained separately for each item apart for the respondents — girls 
and respondents — boys in each of these countries. 
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Figure 2: Ordinary items’ average score and interval score based on the factor COUNTRY. 
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Figure 3: Ordinary items’ average score and interval score based on the COUNTRY and GENDER 
factors. 
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Nominal Item Research Results 


The null hypotheses to each nominal item were statements that the answer on the relevant item 
does not depend on the factor COUNTRY / GENDER / COUNTRY*GENDER. Most of the null hypotheses 
were confirmed. The differences between the respondents’ responses in dependence on some of the 
observed factors were statistically significant only in following cases (table 4): 

e item P7 in relation to the factor GENDER, 

e item P11a in relation to the factor COUNTRY, 
e item P12 in relation to the factor COUNTRY, 
e item P14 in relation to the factor COUNTRY, 
e item P14 in relation to the factor GENDER. 


Table 4. Results of chi-square test of independence for items with proved differences in depend- 
ence on factor COUNTRY or GENDER. 


er Contingency Cramer 

Pearson’s chi-square test td ae 

coefficient coefficient 
ITEM / FACTOR 
e SV p ‘6 ig 

P7 (5) / GENDER (2) 12.41715 4 0.0145 0.18067 0.18369 
P11a (2) / COUNTRY (3) 7.01785 2 0.02993 0.09449 0.09491 
P12 (7) / COUNTRY (3) 146.54410 12 0.0000 0.39834 0.30709 
P14 (7) / COUNTRY (3) 45.77430 12 0.0000 0.23558 0.17141 
P14 (7) / GENDER (2) 13.93940 6 0.0303 0.19104 0.19462 


Figure 4 shows the differences of the responses to the item P12 in dependence on the COUNTRY 
factor, which is the case of the highest significance level. 

Graphs in figure 5 present a summary of the responses to the item 14 according the factor COUNTRY 
and GENDER, which was a case with proven dependence on the both factors. 
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Figure 4; Interaction graph for the item P12 according to the factor COUNTRY. 
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Figure 5: Interaction graph for the item P14 according to the factor COUNTRY and GENDER. 


Discussion 
Discussion of the Ordinary/Scale Item Research Results 


The results presented in figure 2 show that the level of informatics teaching from the student's 
perspective at almost all observed factors is rated the highest in the Slovak Republic. The only two 
exceptions are items P4 and P5, which in case of Slovak respondents obtained significantly low- 
est scores (4.2 for P4 and 5.3 for P5). Slovak students rated informatics as a slightly more difficult 
subject (P4) than the Czech and Belgian students. The average Slovak student rated this subject as 
neither demanding nor easy, while Czech and Belgian students rated it as rather easy. This may be 
closely related to the item P5 results, in which students reflected on the clarity of the teacher's pre- 
sentation of new material. While the Czech and Belgian students usually understand their teacher's 
presentation, Slovak teachers often present material not just appropriately, so students are not 
entirely clear on what is being presented to them (the mean score of the Slovak respondents’ P5 
rating was 5.3, i.e. the rating / rather understand than don’t understand). This factor — the clarity 
of presentation of new material by teachers — might be seen, on the background of our research 
results, as a weakness in informatics teaching in the Slovak Republic. However paradoxically teach- 
ers in Slovakia, in comparison with Czech and Belgian teachers, are able to engage their students 
the most (see the outcome of the item P6). Respondents rated the attractiveness of the teacher’s 
presentation of the Slovak class curriculum with an average score of 5.1. This means that Slovak 
teachers present the material in such a way that students, according to our rating scale, rate it as 
a rather interesting way of presenting curriculum. Belgian students rate the presentation of new 
material by their teachers between neither interesting nor uninteresting and rather interesting (final 
average score was 4.7). From this aspect, informatics teachers in the Czech Republic were rated as 
the worst of the three countries. According to the Czech respondents, the way teachers present the 
subject matter is rather uninteresting (the value of the final average score was 3.3). An analogous 
situation exists with regard to the evaluation of the engagement level of tasks that teachers solve 
with their students during informatics classes in the three countries we studied (see the results 
of the questionnaire item P8). Slovak teachers - again, paradoxically, judging by the results of the 
questionnaire item P5, which states that students only rather understand their presentations — give 
students tasks they considered them to be rather interesting. ln comparison with Slovak teachers, 


243 


—<LLY 


Journal of Baltic Science Education, Vol. 12, No. 2, 2013 


RESULTS OF PISA AND EVALUATION OF COMPUTER SCIENGE EDUCATION ISSN 1648-3898 
(P. 234-248) 


Czech and Belgian informatics teachers obtained significantly worse ratings even in this aspect of 
their assessment (see results for the items P6 and P8). The tasks assigned by these teachers weren't 
considered to be interesting (mean score CZ - 4.0, BE - 4.2 represents the rating neither interesting 
nor uninteresting). 

The knowledge students gain in informatics classes is considered to be rather necessary by both 
Belgian and Slovak students. A significantly more negative outcome was recorded in the evalua- 
tion of computer science education in the Czech Republic. Czech respondents ranked knowledge 
taught in informatics school curriculum as neither necessary nor unnecessary more often than rather 
necessary (P10 average score 4.7). Based on the aspect of attractiveness (P3 questionnaire item), 
the curriculum covered in informatics classes was rated by all three groups as interesting. Also 
noteworthy is the fact that the higher is the average rating by a group, the smaller is the variance 
of the majority of answers (see figure 2 for the minimum size of variance in the majority of answers 
for the group of Slovak respondents, for which the average score in this item was 5.3 - the highest 
one of the three groups, and greater variance of the majority of responses in the case of the Belgian 
respondents, for whom the average score was 4.7 — the lowest one). 

The weakest aspect of teaching informatics at the upper secondary level (ISCED 3A) in all three 
countries has proven to be the quality of textbooks, or rather, their scarcity. The final evaluation of 
the textbook clarity factor (item P9), or the quality of textbooks used in a broader sense, certainly 
cannot be taken lightly, especially given the size of the examined sample and student diversity of 
the residential schools surveyed. Therefore, if we are to formulate recommendations for practical 
utilization of our research results in practice with the goal to increase the level of computer sci- 
ence education (informatics teaching) at schools, the main attention should be drawn to this area. 
In light of the assurance of proper quality of computer science/informatics education at schools, 
results from all three countries showed the need for a deeper analysis of textbooks. However, we 
must point to the fact that the problem of ensuring the quality of textbooks may, in some cases, 
actually be a problem of ensuring the availability of any textbooks. 

On the background of the problems with textbooks, we can rate the performance of informatics 
teachers very high in all three countries, but mainly in Slovakia. Despite the highly unfavourable 
textbook situation, students rate the performance of teachers, in regard to the clarity of curriculum 
interpretation, as very high (see figure 2 — results of the item P5). In all three countries, it is precisely 
this factor that has the highest score. What is more, the Slovak teachers obtained relatively high 
ratings pertaining to the evaluation of their presentation of new material in terms of attractive- 
ness. Only slightly worse, but still good, is their score in the category assessing the tasks they give 
students during class (see figure 2 — items P6 and P8). An interesting result is the evaluation of 
informatics teachers in the Czech Republic. Their performances in terms of providing an intelligible 
interpretation of the new curriculum to students can basically be described as their strong feature. 
But on the other hand, attractiveness of the presentations of new material given by them is, among 
the factors studied, clearly identified as a weakness of informatics education. Moreover also the 
engagement of tasks, which are implemented during informatics lessons by Czech teachers, was 
identified as a weakness (figure 2 — item P8). This factor is a weakness of informatics education in 
Belgium as well. 

Despite the fact that respondents in all three countries recognize the importance of informatics 
knowledge acquisition for everyday life and as an integral part of education (see the results for the 
item P2), deficiencies in the attractiveness of curriculum presentation, uninteresting tasks solved, 
and especially the challenges faced by the lack of proper quality books all these contribute to the 
fact that informatics as a subject is, in all three countries, considered to be a subject, which is nei- 
ther popular nor unpopular, or rather, only more or less popular. On the other hand, in the context of 
continually retrogressive attitudes towards science-oriented subjects, this result can be regarded 
as essentially a positive one. 
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Discussion of the Nominal Item Research Results 


Although the intersexual differences in relation to the preferred ways of new subject matter 
explanation (P7) were proved, the significance of the difference between the responses of the two 
groups is statistically minimal as it shows the value of contingency coefficient (0.18). Most of the 
boys and girls identically prefer explanation done by the teacher himself using various teaching aids 
(alternative b - shared by 48.6 % of the total number of 222 boys and 62.3 % of the total number of 
146 girls). The way of explanation, when the teacher involves also students in it (alternative c),was 
marked by 30.6 % boys and 25.3 % girls. 

A very negative finding is that except computers, any other teaching aids are used very rarely 
in teaching informatics, and this regards all three concerned countries. In the Slovak and Belgian 
group, this response was given more or less by the same percentage of respondents (51.3 %, and 
51.9 %). A significantly higher percentage of negative responses are in case of the respondents 
from the Czech Republic (61.0 %). Given the criticism faced by Slovak teachers following the results 
of the PISA international monitoring, this can be in principle a positive finding for Slovak teach- 
ers, as they use teaching aids on a level comparable with the situation in Belgium and even ona 
significantly higher level than teachers in the Czech Republic (Capay, Magdin, & Tomanova, 2012). 
For all that, the Belgian education system is rated higher than the education system in Slovakia or 
in the Czech Republic, and the funds earmarked for the education sector in Belgium are higher in 
comparison with Slovakia or the Czech Republic. 

The minor part of the respondents (44.7 % from the total number), who stated that their 
informatics teachers use also other teaching aids than computers in their lessons, in the second 
part of the questionnaire item P11 assessed how interesting for them are these other used teach- 
ing aids (P11b). The distribution of negative and positive critical reviews given by the respondents 
was very asymmetrical with a predominance of positive responses (from rather interesting - 9.9 % 
to very interesting — 3.2 %), very high frequency of the neutral statement occurrence (neither un- 
interesting nor interesting — 11.6 %) and the highest frequency of the general positive assessment 
(interesting — 12.5 %). 

The highest measure of interdependence showed the item P12, specifically in dependence on 
the factor COUNTRY (table 4 - contingency coefficient 0.39834). The recorded frequencies (figure 
4) showed that students in Slovakia and Belgium make their written notes mainly in a traditional 
way. The most frequent answers (SK — 60.9 %, BE - 61.6 %) were alternatives a and b (a - the teacher 
dictates us the notes; b - we do our written notes according the teacher who writes the notes on the 
blackboard or screens the notes in an electronic way using a data projector). A little less frequency was 
recorded at the answer c (a part of the written notes we do according the notes made by the teacher and 
a part of the notes we do from the textbooks ourselves). These results show that in general students 
at school are not led to a self-active writing their notes of the studied subject matter. A little bit 
more positive situation is in the Czech Republic where the possibility d (we make our written notes 
completely ourselves on the basis of the teachers’ explanation) was chosen by the highest number of 
the respondents (42.5 %). The answer g (we do not make written notes of the presented subject mat- 
ter at all) was recorded by 21.9 % of the Slovak respondents, 33.3 % of the Czech respondents and 
30.8 % of the Belgian respondents. But this situation is influenced also by the above-mentioned 
fact that the teachers do not very often have suitable textbooks of the required quality and they 
use alternative teaching materials prepared by them or they utilize internet sources. 

On the basis of the research results one can say that the ways used at schools to take written 
notes regarding informatics subject matter are suitable and appropriate for students, they are 
satisfied with them and do not want to change them, i.e. in each concerned country the students 
are satisfied with the way which is more or less specific just for their country (p = 0.13000 means 
that assessment of the items P13a does not depend on the factor COUNTRY, the same result was 
obtained also in tests of the item P13a dependence on the factor GENDER). Only one fifth (19.8 %) 
of the total number of the respondents is unsatisfied and would prefer another way. 
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Asking the students about source of concern related to informatics we expected occurrence 
of students’ rather positive responses to this item what was more or less proved as 51.7 % of the 
total number 368 respondents chose the offered alternative a (! am not used to be afraid of anything). 
It was the most frequent answer in all three groups of the respondents. In Slovakia and the Czech 
Republic this answer was followed by the answer b (unpreparedness, i.e. |am not prepared properly) 
given by 17.4 % of the Slovak respondents and 12.6 % of the Czech respondents. In the sample of 
Belgian students the second most frequent answer was the alternative d (practical tests) marked by 
21.2 % of the respondents. The results show that the Slovak students, unlike the Belgian ones, are 
not afraid of any theoretical or practical testing of the acquired knowledge. They are rather afraid 
of being not prepared properly for the lesson what evokes at them fear of threat of a bad mark 
getting if the teacher examines them (figure 5). Beside the dependence on the factor COUNTRY, the 
answers to the item P14 showed a trivial degree of dependence also on the factor GENDER (table 
4). Although the two most frequent answers were the same in each group (54.5 % mail respondents 
and 45.2 % female respondents declared that before the informatics lessons they are not used to 
be afraid of anything, and 18 % male respondents and 16.4 % female respondents declared a weak 
preparedness as a usual source of their fear before the informatics lessons), the rest of possible 
reasons of the students’ fear before the informatics lessons proved significant differences between 
boys and girls (the second graph in figure 5). In case of girls the fear of examination is markedly 
higher. 9.6 % of female respondents are afraid of oral examination and 13 % of practical examina- 
tion, whereas in case of male respondents these percentages are lower, identically equal to 5.4 % 
for the both types of examination. On the other hand boys are more afraid of getting a bad mark 
(9.5 %) than girls (4.8 %). 


Conclusions 


Based on the presented research results, we can identify strengths and weaknesses of infor- 
matics teaching at the upper secondary level of education in each concerned country. 

In Slovakia the main weakness seems to be insufficient quality, or rather the lack of relevant 
textbooks. The results also indicate a large deficit in terms of the attractiveness of tasks that teachers 
use during informatics lessons. On the other hand attractiveness of the curriculum and methods of 
new material presentation by teachers (in terms of clarity and engagement) are evaluated positively 
from the point of students’ view. 

Opposite to Slovakia, in the Czech Republic the teacher's presentation of new material in terms 
of its attractiveness and level of engagement for students was identified as the weakest aspect. 
The level of interest in tasks assigned during class was also rated as a weakness (in Slovakia this 
item has not been clearly identified as a weakness but as an area with considerable deficiencies). 
Similarly to the Slovak Republic also in the Czech Republic the insufficient quality, or rather the 
lack of appropriate textbooks belongs to weaknesses of informatics teaching. The clarity of the 
teacher's presentation of new curriculum can be described as strength. 

In Belgium, the weakest aspect of informatics teaching is the level of engagement, or rather 
the lack thereof, in tasks covered in class. Another weakness, similarly to Slovakia and the Czech 
Republic, is the insufficient quality and the lack of textbooks. The strength of informatics teaching 
is the clarity of a teacher’s presentation of new material. 

Textbooks have proven to be a weakness in the implementation of teaching informatics in all 
three countries. To solve this problem means to solve the question of textbook existence, assessment 
of the adequacy and level of textbooks used, or rather, the assessment of teaching materials used 
in those books, analysis of needs arising from everyday life and the design of appropriate ways of 
dealing with this issue. Although it isa common problem of the concerned countries, it cannot be 
solved globally and must be analysed separately in each of them. 

In carrying out our research, our options in creating individual research samples were limited. 
Moreover the carried out evaluation has been based only on the evaluation of the selected factors 
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affecting the quality and attractiveness of computer science education only from the students’ 
point of view (i.e. the teachers and other experts’ points of view are missing). We realize that due 
to these facts the findings of our research cannot be taken as completely general. Despite that, we 
believe that our results showed the existence of both certain negative and positive phenomena 
in teaching informatics in each concerned country and can contribute to increasing the quality 
of informatics teaching and computer education, and consequently also to the development of 
scientific literacy and technical education in each of them. 
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